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Abstract 

Aim: Evaluation of patient satisfaction constitutes an important step in improving the quality of care. Because of difficulties in evaluating patient satisfaction 
in the Emergency department, a practical method specific to the emergency department is needed. In this study, the Turkish validity and reliability trial of 
BEPSS, which evaluates patient satisfaction in the emergency department, was carried out. 

Material and Methods: This is a scale validity and reliability study. First, BEPPS was translated into Turkish. The scale was then tested on 200 patients. Based 
on the data obtained, the construct validity, criterion validity and the reliability of the scale were evaluated. 

Results: The average total score of BEPSS was 73.50+10.345. In the explanatory factor analysis for construct validity, the eigenvalue of a factor was found to 
be 5.282, and the total variance was 75.46%. According to the Cronbach alpha coefficient used in the reliability analysis, the internal consistency of the scale 
was 0.79. In the correlation matrix, it was seen that there was a positive and average correlation between the items of the scale. 

Discussion: In the study, it was found that the Turkish version of BEPSS was appropriate in terms of criterion validity, structural validity, and reliability. 


Keywords 
Patient Satisfaction; Emergency Department; Validity; Reliability; Scale 


DOI: 10.4328/ACAM.20587 Received: 2021-03-14 Accepted: 2021-06-07 Published Online: 2021-06-19 Printed: 2021-09-01 Ann Clin Anal Med 2021;12(9):1058-1062 
Corresponding Author: Hiiseyin Acar, Department of Emergency Medicine, Izmir Katip Celebi University Atatiirk Training and Research Hospital, izmir, TURKEY. 

E-mail: dracar@hotmail.com _P: +90 (532) 500 90 29 

Corresponding Author ORCID ID: https://orcid.org/O000-0002-1905-7133 


Annals of Clinical and Analytical Medicine 1058 


Turkish adaptation of brief emergency department patient satisfaction scale 


Introduction 

Patient satisfaction is a concept directly related to the quality 
of the healthcare provided to patients [1]. The increase in 
the value given to the patient and the quality of healthcare 
affect patient satisfaction positively [2]. Due to the increasing 
competition in the health sector, patient satisfaction and 
health service quality have become increasingly important in 
recent years [3]. There are many different factors that affect 
patient satisfaction [4]. Factors affecting patient satisfaction 
include the number of healthcare professionals in the hospital, 
bed capacity of the hospital [5], patient s age, communication 
between the patient and the healthcare provider [6], waiting 
time of the patients, mutual empathy, respect for patients and 
their relatives, and patient privacy [7]. 

Patient satisfaction surveys used to evaluate the level of patient 
satisfaction often focus on the hospital environment, the health 
care provided to the patient, and the attitude of hospital staff 
[4]. However, features such as Emergency department crowding 
[8], the urgency of the patient’s condition, long waiting times, 
and anxiety of patients or their companions, which distinguish 
the emergency department from other departments, can 
directly affect patient satisfaction [9]. For this reason, it will 
be appropriate to use a specialized patient satisfaction survey 
for the emergency department. Atari et al. developed the “Brief 
Emergency Department Patient Satisfaction Scale (BEPSS)’, 
which evaluates patient satisfaction in different aspects for 
the emergency department. BEPSS is a patient satisfaction 
questionnaire specific to the emergency department, as it is 
prepared based on emergency department-specific problems. 
This scale, prepared in English, questions the satisfaction level 
of the patients through 20 questions under 5 subgroups [10]. 
This study aimed to translate BEPSS into Turkish and conduct 
the validity and reliability study of BEPSS for Turkish society. 


Material and Methods 

Study Design and Ethics 

This study is a scale validity and reliability study carried out 
prospectively. Study was approved by the local ethics committee 
(2019-GOKAE-1096). 

Study Setting 

This study was conducted in the ED of a training and research 
hospital with 350,000 annual ED visits. 

Patient Selection 

Patients aged 18 and over who presented to the emergency 
department and volunteered to participate were included in 
the study. Patients with a history of alcohol or drug use within 
the last 48 hours, patients who were unconscious or intubated, 
patients who refused to participate in the study, and patients 
who did not complete the questionnaire were excluded from the 
study (Figure 1) 

Data collection 

A single questionnaire including the Brief Emergency 
Department Patient Satisfaction Scale (BEPSS), demographic 
data of the patients and the Numeric Rating Scale (NRS) that 
measures the patient’s satisfaction level was used to collect the 
data. BEPSS consists of 5 subgroups and a total of 20 questions 
evaluating patient satisfaction in the ED. The subgroups of the 
scale are emergency department staff (EDS) consisting of 6 


questions, emergency department environment (EDE) consisting 
of 3 questions, physician care satisfaction (PCS) consisting of 
4 questions, general patient satisfaction (GPS) consisting of 5 
questions and patient’s family satisfaction (PFS) consisting of 
2 questions. BEPSS is a Likert-type scale with each question 
scored between 1 and 4. Among the points given in response 
to the questions, 1 point means completely disagree, 2 points 
mildly disagree, 3 points mildly agree, and 4 points completely 
agree. By collecting the points given by the patient to each 
item, the patient’s emergency satisfaction score is obtained. 
While the minimum score that can be obtained from the whole 
scale is 20, the maximum score is 80. While a score of 20 points 
taken from the scale indicates complete dissatisfaction of the 
patient, a score of 80 points indicates complete satisfaction. 
The demographic data collection form includes age, gender, 
time for presentation to the emergency department, waiting 
time before seeing the doctor, patient’s triage category, 
hospitalization status of the patient (outpatient/hospitalization/ 
intensive care), previous medical history, educational status, 
and smoking information. The Numeric Rating Scale (NRS), 
which measures the level of satisfaction in order to compare 
with BEPSS, is scored between O and 10, while “O” means 
complete dissatisfaction, “10” means complete satisfaction 
[11]. Verbal consent was obtained from all patients before the 
questionnaires were filled. The questionnaire was completed 
by the patients themselves. For elderly or visually impaired 
patients, the questionnaire form was read out loudly by the 
researcher, and the patient’s responses were written on the 
form by the researcher. However, in conditions where it was not 
possible for the patient to complete the questionnaire for any 
reason, one of the patient’s relatives was asked to complete 
it. The questionnaire was completed just before the patient 
left the ED. For any patient who needed immediate treatment 
for a life-threatening condition, after the patient’s treatment 
was completed, one of the relatives of the patient was politely 
asked to complete the questionnaire form. 

Preparation of the Turkish Version of the Scale 

Firstly, in order to prepare the Turkish adaptation of the scale, 
permission to use the scale was obtained by e-mail from 
Mohammad Atari, the developer of the original version of 
BEPSS. Turkish adaptation of the scale, completed in 6 steps, 
was prepared based on the guideline developed by Beaton et 
al. [12]. 

First step: The scale was translated into Turkish separately by 
an emergency medicine specialist who knew the concept of the 
study, whose native language was Turkish and who could speak 
English well, as well as an English foreign language lecturer who 
was not familiar with the concept and whose native language 
was English. 

Second step: Both the Turkish translations of the scale and the 
original English version of the scale were synthesized by two 
translators, accompanied by an observer, and a single Turkish 
version was created. 

Third step: The Turkish version of the scale has been 
retranslated into English by two translators whose native 
language was English and who spoke Turkish well and were 
blinded to the original version of the scale. 

Fourth step: This is the step where the translations were 
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evaluated by a committee and the Turkish version of the scale 
was finalized. In this step, the meaning compatibility between 
the Turkish version of the scale and the original English version 
was evaluated. The committee included an emergency medicine 
specialist and an English lecturer who were translators of the 
original scale into Turkish, and two translators who retranslated 
the scale back to English. 

Fifth step: In this step, the Turkish version of the scale, approved 
by the committee, was evaluated for clarity in a pilot study of 
30 participants. The volunteers who filled the questionnaire 
were asked to evaluate each item on the scale for clarity. In this 
pilot study, it was determined that there was no item that could 
not be understood by the patients. 

Sixth step: In this step, the version of the scale retranslated 
from Turkish to English and the final Turkish version of the 
scale were sent to Mohammad Atari, the developer of the scale, 
and his approval was obtained. 

After the 6-step stage in which the Turkish version of the 
scale was prepared, the validity and reliability study of the 
Turkish version of BEPSS was performed. In this study, patients 
presented to the emergency department were asked to fill 
in the questionnaire after verbal consent was obtained. The 
research was terminated when the targeted 200 participants 
were reached. The Turkish validity and reliability of BEPSS were 
investigated as described in the statistical analysis section. 
Sample Size 

It is stated in the literature that a sample of 10 times the 
number of items on the scale is ideal for scale validity and 
reliability studies [13]. Since there were 20 items on this scale, 
the sample size was determined as 200. 

Statistical Analysis 

The data obtained in the study were analyzed using the SPSS 
20.0 package program for Windows (IBM Corporation, Armonk, 
New York, USA). While evaluating the study data, the frequency 
distribution (number, percentage) for categorical variables and 
descriptive statistics (mean, standard deviation) for numerical 
variables were used. In statistical analysis, Type-1 error was 
taken as a = 0.05 for statistical significance. The Cronbach 
Alpha Reliability Coefficient, which is one of the internal 
consistency methods, was used in the reliability study. The 
Kaiser-Meyer-Olkin (KMO) coefficient and Barlett tests were 
used to evaluate the feasibility of factor analysis, and factor 
analysis is used to evaluate the suitability of the construct 
validity of the scale. For criterion validity, BEPSS’s correlation 
with the NRS satisfaction scale was examined. The relation 
between subgroups of the scale was examined using the 
Spearman Correlation analysis. 


Results 

Socio-Demographic Characteristics of Patients 

A total of 200 patients participated in the study. The average 
age of the patients was 46 + 20.92 years, and 46% were 
women. The triage category of 102 (51%) patients was green, 
70 (35%) were yellow and 28 (14%) were red. One hundred 
fifty-seven (78.5%) of the patients included in the study were 
discharged from the emergency department, 133 (16.5%) were 
hospitalized and 10 (5%) were admitted to the intensive care 
unit. 


The average total BEPSS score was 73.50 + 10.345 and the 
average NRS satisfaction score was 8.84 + 1.75. Looking at the 
average scores of the BEPSS subgroups, average EDS score 
was 21.80 + 4.030, average EDE score was 10.50 + 2.110, 
average PCS score was 15.41 + 1.821, the GPS scale score 
was 18.40 +3.129 and average PFS score was 7.28 +1.466. 
When the relationship between NRS satisfaction score and 
BEPSS subgroup scores were examined one by one, there was a 
positive and significant relationship between them (p=0.0001). 
In addition, when the patients’ responses to the scale were 
evaluated using the Spearman Correlation analysis, it was 
calculated that the subgroups had a positive relationship with 
each other, as shown in Table 1 (p=0.00001). 

Reliability Analysis 

Cronbach's alpha coefficient was used in the reliability analysis 
where the internal consistency of BEPSS was evaluated. When 
the correlation between subgroup items and the correlation of 
BEPSS with subgroup items were evaluated, the correlation 
coefficients were found to be in the range of 0.56-0.80 
(p=0.0001), and the internal consistency of the scale was 0.79. 
Validity Analysis 

The Kaiser-Meyer-Olkin (KMO) coefficient, which evaluates 
the adequacy of the sample size, was found to be 0.64 and 
the Bartlett Test (x2 = 1782.42, SD = 12, p=0.00001), in which 
the suitability of the scale was evaluated was found to be 
significant. In the explanatory factor analysis for construct 
validity, the eigenvalue of a factor was found to be 5.282, and 
the total variance was 75.46% as shown in Table 2. 

When the relationship between the items of the scale was 
examined in the correlation matrix, it is seen that there is a 
positive and average relationship, as shown in Table 3. Since it 


is the only factor, Varimax rotation cannot be done. 


The Number of Patients Eligible 


Patients exluded 
Alcohole use:3 
-Drug use:4 
-Under 18 old: 21 
-“Uncooperated:4 
-Total:32 


N:360 


The Number of Patients 
Recruited 
N:328 


Number of patients 
did not agree to 
partipicate:110 


The Number of Patients 
Consented 
N:218 


Number of patients 
excluded for 
missing data:18 


The Number of Patients 
Enrolled 
N:200 


Figure 1. Patient Flow Chart 


1060] Annals of Clinical and Analytical Medicine 


Turkish adaptation of brief emergency department patient satisfaction scale 


Table 1. Correlation Coefficients of BEPSS, BEPSS Subgroup Items and NRS Satisfaction 


NRS 


satisfaction 


EDS 1.000 

EDE 0.530 1.000 

PCS 0.468 0.493 1.000 
GPS 0.465 0.569 0.455 
PFS 0.560 0.512 0.514 
BEPSS 0.779 0.803 0.568 
NRS Satisfaction 0.706 0.767 0.548 


Brief Emergency Patient Satisfaction Scale, NRS: Numeric Rating Scale 


Table 2. Factor analysis for BEPSS 


Total value 


Component 


% (Variance) % (Total) 


EDS 5.282 75.454 75.454 
EDE 0.525 7.501 82.955 
PGs 0.428 6.119 89.074 

GPS 0.390 5.571 94.645 

PFS 0.272 3.881 98.526 
BEPSS 0.094 1.339 99.865 
NRS Satisfaction 0.009 0.135 100.000 
EDS: Emergency Department Staff, EDE: Emergency Department Environment, PCS: Physi- 


cian Care Satisfaction, GPS: General Patient Satisfaction, PFS: Patient's Family Satisfac- 
tion, BEPSS: Brief Emergency Patient Satisfaction Scale, NRS: Numeric Rating Scale 


Table 3. Correlation Matrix Investigation to Evaluate the Con- 
tribution of the Subgroup Items to the Scale 


ers Beier 
EDS 1.000 
EDE 0.605 1.000 
PCS 0.641 0.569 ‘1.000 
GPS 0.591 0.714 0.634 1.000 
PFS 0.533 0.594 0.526 0.605 1.000 
BEPSS 0.846 0.817 0.803 0.874 0.736 1.000 0.932 
NGS 0.808 0817 0.716 0.787 0.707 0.932 1.000 


Satisfaction 
EDS: Emergency Department Staff, EDE: Emergency Department Environment, PCS: Physi- 


cian Care Satisfaction, GPS: General Patient Satisfaction, PFS: Patient’s Family Satisfac- 
tion, BEPSS: Brief Emergency Patient Satisfaction Scale, NRS: Numeric Rating Scale 


Discussion 

In this study, the Turkish version of BEPSS developed by 
Atari et al. evaluating patient satisfaction in the emergency 
department was prepared, and its validity and reliability study 
was performed. Within the scope of the study, firstly, Turkish 
version of the scale was prepared and then criterion validity, 
construct validity, internal consistency and explanatory factor 
analysis were evaluated. It was seen that the scale was valid 
and reliable and it was appropriate to be used inTurkish society. 
Validity Analysis 

The purpose of the validity analysis is to examine whether the 
items of the scale represent the area or behavior desired to 
be measured, and to form an entirety of meaningful items by 
a group of experts [14,15]. The only condition for measuring 
criterion validity is that there is a suitable and valid criterion 
to compare. The aim is to establish the relationship between 


1.000 0.0001 
0.577 1.000 

0.734 0.678 1.000 

0.711 0.610 0.897 1.000 


EDS: Emergency Department Staff, EDE: Emergency Department Environment, PCS: Physician Care Satisfaction, GPS: General Patient Satisfaction, PFS: Patient's Family Satisfaction, BEPSS: 


the valid measurement tool and another measurement tool 
[16]. In this study, concurrent validity was used to measure 
criterion validity. In concurrent validity, the score obtained from 
the scale to be tested is compared with the score obtained 
from another scale, measuring the same or similar behavior. 
The tests to be compared should be carried out simultaneously 
or within a close time [17]. In this study, NRS satisfaction and 
BEPSS were performed simultaneously. A positive correlation 
was found between BEPSS and NRS satisfaction. According to 
these results, the criteria validity of BEPSS has been confirmed. 
The Kaiser-Meyer-Olkin (KMO) coefficient and Barlett tests are 
used to evaluate the feasibility of factor analysis [18]. The KMO 
coefficient must be over 0.50 for factor analysis. In addition, the 
interpretation of the KMO coefficient between 0.70 and 0.80 is 
moderate, between 0.80 and 0.90 is good, and above 0.90 is 
excellent sampling [19]. The Bartlett test is used to evaluate the 
suitability of items on the scale for the factor analysis of the 
correlation matrix [18]. In this study, the KMO coefficient was 
determined as 0.64 and the Bartlett test was significant. In the 
light of these findings, it was concluded that the sample size for 
the study was sufficient and feasible for factor analysis. 
Factor analysis is used to evaluate the suitability of the construct 
validity of a scale. When conducting exploratory factor analysis, 
it is necessary to take into account whether the scale has a 
single or multi-factor structure. In single-factor scales, the 
scale is expected to explain at least 30% of the total variance, 
while in multi-factor scales this value should be even higher 
(20). In the exploratory factor analysis evaluating BEPSS, when 
the items of the scale were compared through factor analysis, 
it was found that the initial eigenvalue of one factor was 5,282 
and the total variance was 75,46%. Since this value explains 
the variance of the scale, the scale can be explained using a 
single factor. According to these results, it can be said that the 
structural validity of the scale was confirmed. 

Reliability Analysis 

Reliability analysis means that the scale can reflect the structure 
it measures consistently. In reliability studies, Cronbach's alpha 
value is used while evaluating internal consistency, which 
shows that the scale items are consistent with each other and 
contribute to the scale in the same direction. In order to say 
that the scale is reliable, Cronbach's alpha value must be above 
0.70 [21]. In this study, Cronbach's alpha value was found to be 
0.79 in the internal consistency analysis for BEPSS. According 
to this result, it can be said that the Turkish version of BEPSS 
is reliable. 
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Conclusion 

This study shows that the Turkish version of BEPSS is an 
easy-to-use, reliable and valid instrument that can be used to 
evaluate patient satisfaction in the emergency department. 
Patient satisfaction assessments that will be performed 
in the emergency department using BEPSS will enable the 
identification of problems specific to the emergency department 
and the improvement of health care provided to the patients. 
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